
The Journal of Effective Teaching 

an online journal devoted to teaching excellence 


Benchmarking Quality in 
Online Teaching and Learning: 

A Rubric for Course Construction and Evaluation 

Mona P. Temus 1 

Associate Professor, University of New Mexico, New Mexico 

Kay L. Palmer 

Associate Professor, Old Dominion University 

Debbie R. Faulk 

Distinguished Teaching Associate Professor, 

Auburn University Montgomery, Alabama 


Abstract 

Online courses have many components and dimensions. Both the form (stmcture) and 
the content (expression) are situated in an overall environment. The sum of these 
elements results in student outcomes and learning. In order to facilitate constmction and 
evaluate the quality of an online course, a four-part rubric was designed to reflect: 

Structure (Context, Organization, and Environment) 

Content (Presentation of Information) 

Processes (Relationships and Interactions) 

Outcomes (Mastery of Content and Course Evaluation) 

This mbric was designed to provide quantitative and qualitative standardized evaluation 
for faculty self-evaluation, peer evaluation, and administrator evaluation. The rubric was 
piloted at two universities and shown to be highly effective in eliciting effective and 
usable feedback for course instructors and program directors. It was concluded that a 
uniform rubric that can be applied to any discipline could facilitate evaluation of all 
online courses within a program to a set standard that can then be used for course 
enhancement and improvement with structured comprehensive evaluation from 
instructors, peers, or program directors. It was found that a well-designed course 
(structure), with relevant and credible information (content), as well as mechanisms for 
interaction and collaboration (processes), could result in enhanced student learning 
(outcomes). 

Keywords: Distance Education, Online Learning, Instmctional Design 


Web-based, or online, teaching is grounded in cyberspace and allows students the 
flexibility to learn anytime and anyplace, and at a time when they choose to focus 
on the course content. It gives the student and the teacher time for reflection and a 
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means for all to participate and interact. This opportunity eliminates many of the 
barriers related to traditional classroom learning. A majority of the research 
related to the effectiveness of online learning demonstrates that there are few 
differences in outcomes. In fact, it is purported that Web-based learning is “just as 
good” as traditional, face-to-face instruction (Allen & Seaman, 2006, Armstrong, 
Gessner, & Cooper, 2000; Herther, 1997). 

There are two issues that must be addressed in online courses. The first is the 
quality of the teaching tool and the second is the quality of the learning that takes 
place. The quality of learning from online courses is well established (Allen & 
Seaman, 2006; Buckley, 2003; DeBourgh, 2003) and the benchmarks for quality 
of the online course are available in a myriad of sources (Billings, 2000; Jairath & 
Stair, 2004; Phipps & Merisotis, 2000; Richard, Mercer, & Bray, 2005). The 
rubric instrument developed by the authors and described in this article can be 
used as an evaluation tool to determine if your course maximizes technology in 
course construction to enhance quality pedagogy. This rubric is different than 
those previously developed (Keinath & Blicker, 2003; Wolf & Stevens, 2007) as 
it can be used to assess both course construction and learning outcomes. 

Growth of Online Learning 

The growth of online learning has been rapid and phenomenal. From the early 
correspondence model of distance education provided by the U.S. postal system 
in the 1800s to today’s campus portal access using multimedia, Internet, and 
computer media communications, distance learning has evolved into a 
technology-driven, student-demanded market. An estimated 
3.2 million students are using online learning technology, substantially increased 
from the previous year of 2.3 million students; and universities are reporting that 
online learning is critical to long-term strategies (Allen & Seaman, 2006). Palloff 
and Pratt (2001) reported that almost 90% of institutions with enrollments of 
10,000 or more are offering some form of Web-based education. Hosie and 
Schibeci (2005) noted that education and global learning is its own “mega trend.” 
Predictions abound regarding the virtual university of the world without any 
national boundaries (Moe & Blodget, 2000; Taylor, 2001). There should be no 
doubt that online learning is vital to all disciplines involved in education in the 

St 

21 century. 


Online Learning in the Health Sciences 

A call for the health sciences to use Internet technology as a tool for delivering 
education has been issued for some time now (Cobb & Baird, 1999; Franck & 
Langenkamp, 2000; Thurmond, Wambach, Connors, & Frey, 2002). Although 
one would be hard pressed today to find a medical or nursing program that is not 
using some type of Web-based education, educators and students have not readily 
embraced this educational tool (Frase-Blunt, 2000; Monke, 2005/2006; Reynard, 
2007; Schmitt, Titler, Herr, & Ardery, 2004; Sit, Chung, Chow, & Wong, 2004). 
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Research related to online learning is varied and includes comparing learning in 
the traditional classroom with Web-based education, comparing group discourse 
in the “wall-less” classroom with cyberspace learning, and evaluating online 
course development and effectiveness. There is little research related to 
evaluation tools of course construction to support faculty in effectively conveying 
content to students online. In a study conducted by Arbaugh in 2000, students 
involved in Web-based courses actually conversed more than those in a traditional 
classroom. 

Online delivery is a relatively new addition to the educational methods used in 
schools for health sciences. Although much effort has been directed toward 
development of methodology, less emphasis has been placed on the evaluation of 
the strategies used to deliver the content. As programs move to increase the 
course offerings using online delivery, it is imperative that faculty develop a 
systematic method of evaluating the online strategies used in course delivery. 

Issues in Online Learning 

There are many issues swirling around the use of online teaching and learning. 
Some of these include faculty readiness and willingness, administrative and 
infrastructure support, accessibility, student success, costs, efficiency and 
effectiveness issues. Each of these issues is multifaceted and involves the 
development and implementation of specific policies and procedures. Provision of 
a framework that will allow for consistent and coherent technology, software, and 
course design decisions is crucial. For example, can course content be delivered 
online (no face to face contact with teacher) or through a hybrid/enhanced 
medium (one that is partially face to face and partially online). In the hybrid 
course, the online portion must meet “best practices” for online learning, as well 
as, “best practices” for classroom learning. Hybrid courses can be very enticing 
to a student who may not have the time or finances to be on site several days a 
week during a semester but can come at known intervals. The hybrid method may 
also be appealing to teachers who are new to online teaching and may not yet 
“trust” this medium as pedagogically capable. 

Faculty support and effectiveness of online delivery were two of the major 
concerns that led to the development of a rubric that can standardize evaluation of 
an online course. This rubric can be used for course enhancement and 
improvement with structured comprehensive evaluation by the instructor (self- 
evaluation), colleagues (peer evaluation), or deans and directors (program 
evaluation or external evaluation). In this way, faculty can use the rubric to assist 
with both design and evaluation of a course. Part of advancing faculty 
development as online course designers and facilitators is to have a 
conceptualization framework, which allows for a way to visualize the various 
elements that exist in online teaching. The rubric is a framework that can provide 
this means of conceptualization. 
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Rubrics 

A rubric is a model or template that can be used as an evaluation instrument for 
assessment of a body of work aligned with set standards. According to Wolf & 
Stevens (2007) “rubrics improve teaching, contribute to sound assessment, and 
are an important source of information for program improvement” (p. 3). A good 
rubric can facilitate a definition of excellence, communicate exemplary practices, 
communicate goals or expectations, and allow for accurate and consistent 
evaluation of a body of work by documenting the procedures used in making 
judgments. A rubric organizes and clarifies criteria along a continuum in such a 
way that two individuals who apply the rubric to a body of work will generally 
arrive at a similar score. The greater the agreement between the scores assigned 
by two independent assessors is a measure of the reliability and interrater 
reliability of the rubric as an assessment tool. The rubric developed by the 
researchers (Figure 1) was designed to be a general rubric instrument using terms 
that were not discipline-specific. In other words, a conscious effort was made to 
choose terms that were generic to all online courses. 

Development and Testing of the Rubric 

An extensive review of the literature indicated that limited strategies exist for 
evaluation of online course construction and delivery. Keinath and Blicker (2003) 
developed a rubric to assess readiness of online courses prior to course delivery. 
This was used to expedite feedback to instructors and ensure consistency of site 
review, meaning that instructors could use the rubric to review basic elements 
prior to the start of an online course. The rubric was then further tested to identify 
the “student-readiness” of a site. Whereas course readiness is evaluated prior to 
the start date of the course, the rubric described herein expands beyond readiness, 
to include the concepts of content, interactions and processes, as well as 
summative evaluation. 

The process of development of our rubric was creative, innovative and 
straightforward. The three developers agreed to the general goal of the creation of 
a rubric to evaluate overall online course delivery. The first step in the process 
was an evaluation of the literature, discussions with other faculty and students, 
and reflection on courses taught via the Internet. The developers then discussed 
all the information and agreed on a general structure for a rubric with four 
dimensions: (a) Structure, which encompasses context, organization, and 
environment; (b) Content, which encompasses the presentation of information; (c) 
Processes, which encompasses human aspects, relationships, interactions, and 
quality; and (d) Outcomes, which encompasses student learning and mastery of 
content, as well as course evaluation. Each of the major dimensions has several 
components, which can be measured both quantitatively and qualitatively. It was 
hypothesized that a well-designed course (structure), with relevant and credible 
information (content), as well as mechanisms for interaction and collaboration 
(processes), can result in enhanced student learning (outcomes). 
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In examining each of these four overarching dimensions, various elements and 
sub-elements emerged. These elements and sub-elements were discussed over a 
series of teleconferences. As this was a creative process with the developers 
having a thorough understanding of the literature and a combined over 10 years of 
online teaching, it was decided that the best way to synthesize our knowledge and 
experience was to jointly develop the first dimension of “Structure”. The initial 
development of the first dimension involved each developer writing definitions 
for the elements and compiling the findings. Teleconferences were held to refine 
and consolidate the definitions for the first dimension. At this point in the 
development process, the developers individually tested the first dimension, 
which resulted in further refinements in each of the elements, as well as the 
definitions related to the presence or absence of certain attributes. The first testing 
facilitated the development process for the remaining three dimensions. As in any 
creative process, the development of an instrument is not always linear, and many 
times during the development of the rubric, elements and sub-elements of other 
dimensions would emerge from the work. In developing each part of the rubric, 
data from one dimension would also inform the development of elements in other 
dimensions. Refinements to the rubric continued over a period of 6 months. 

The rubric was pilot tested at two universities with a convenience sample. Six 
faculty members from different departments who had more than two years of 
experience in online teaching were recruited to complete the rubrics and offer 
their comments. Recruitment occurred by the developers asking three faculty 
members from different departments at each university site to use the rubric to 
evaluate any online course they were teaching. No formal training on use of the 
rubric was provided. Data gathered from the pilot was primarily qualitative in 
nature, with extensive comments written on the rubrics provided by the 
developers for the pilot testing process. Data were analyzed using a constant 
comparative method to allow for emergent themes. The developers were all 
experienced in online course delivery and thus credible and reliable instruments to 
analyze and interpret the qualitative pilot data. All three developers 
independently evaluated the comments from the pilot data related to both the 
scoring process and the open-ended comments sections. The developers then 
compared the results. Overall the comments reflected that the faculty involved in 
the pilot test thought that the rubric proved to be highly effective in eliciting 
valuable and usable feedback for course instructors and program directors related 
to both course construction and course evaluation. Specific suggestions were 
made for development of various elements and sub-elements within the rubric. 
Refinements were then accomplished by the developers based on the evaluation 
of pilot data. 

The next step was to establish content validity. Using the revised rubric, three 
experts in online teaching, who had greater than five years of experience and were 
not at the two pilot university sites, were recruited to evaluate the rubric for 
content validity. These experts were given the rubric, selected literature, and were 
asked to use their experiential knowledge to ascertain if the rubric was 1) overall a 
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good measure for course evaluation and 2) if the dimensions, elements and sub¬ 
elements were inclusive and valid. Content validity was thus established via this 
review by three external experts. 

At the same time, the developers engaged in a process to determine reliability of 
the rubric. The three developers tested the rubric on two courses they could 
access online, for a total of six courses, then retested one to two weeks later, for 
test-retest reliability. Each developer reported on the overall reliability for both 
the scoring and qualitative remarks. The instrument was found to be reliable. 
Minimal adjustments to the rubric were accomplished after the content validity 
and reliability assessments. 


Evaluation Rubric 

The four-part rubric is divided into the four major dimensions: Structure, Content, 
Processes, and Outcomes. After each section, there is a page allocated to 
comments; and at the end of the rubric, there is a page allocated to comments 
about the overall course. The rubric is presented in Figure 1. 

The most straightforward areas of the rubric with evaluation criteria that is 
evident and visible to most educators are the first and third sections related to 
course structure and processes. A more advanced evaluation is needed of the 
second and last areas, content and outcomes, as these areas require a knowledge 
of the content, a certain amount of expertise in the subject matter, and the ability 
to discern what is most important and relevant in learning the material. To 
effectively evaluate the content presented and student outcomes, the reviewer 
would need to understand the discipline, the subject, and be knowledgeable of the 
current state of the science/art and landmarks within the field. To a certain extent 
this is also true of evaluating the processes and interactions between the faculty 
and the students, although if faculty and students are engaged in dialogue most 
educators can determine if learning and effective communications are present. 
An example of a completed content section of the rubric is presented in Table 1. 

A program administrator can use the rubric to review several courses, or the same 
course over time, with standardized criteria. An example of an instrument grid to 
facilitate a review of the same course during different semesters is presented in 
Table 2. 


Conclusions 

The Online Course Construction and Evaluation Rubric serves multiple purposes: 
it is a mechanism for self-evaluation, peer evaluation, and administrator, program 
director, and/or dean evaluation; it can be used to design or facilitate an online 
and hybrid/enhanced course; and it can be used to enhance the creation of a 
collaborative online learning environment. It was concluded that a uniform rubric 
that can be applied to any discipline could facilitate evaluation of all online and 
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hybrid/enhanced courses to a set standard that can then be used by faculty and 
program directors to promote faculty development and subsequent student 
learning, as well as course evaluation, course design, and the creation of an online 
collaborative learning environment. However, as with any tool, there are 
limitations. Technology and software mediums are rapidly expanding educational 
boundaries. The use and effectiveness of technology such as live video, 
Captivate© and WIMBA© are difficult to evaluate. Partnering with technology 
specialists could provide a fluid framework for the development of essential 
elements related to technological advances. Additionally, any tool is really only 
as good as its user. The more skilled and knowledgeable the person using this 
tool, the more data and better judgments can be elicited. For example, only a 
person with knowledge and expertise on a particular subject can determine if the 
content presented is truly an accurate, in-depth, appropriate treatment of the 
subject matter and if the assignments, interactions, and evaluations are substantive 
and conducive to learning. 

Advances in technology and demands from consumers are driving changes in 
educational methodologies. Online learning will continue to expand and be 
recognized as a valuable educational tool. Institutions and educators from all 
disciplines must keep pace with these changes by providing a learning 
environment that will meet the demands of consumers and stakeholders. 
Development of effective methods for evaluation of online courses is an 
important step in meeting the challenges of online teaching and learning. 
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Figure 1: 


Online Course Construction and Evaluation Rubric © 


INSTRUCTIONS: 

There are four major dimensions to the online course evaluation rubric: 

> Structure — Conte xt/Organizati on/Environment 

> Content — Presentation of Information 

> Processes - Human Aspects/Relationships/Interactions/Quality 

> Outcomes — Mastery of Content and Course Evaluation 


There are both quantitative and qualitative measures. Each element within the overall dimension can be scored with 0 to 3 points, with a 
numerical total summed for each dimension. Qualitative evaluation can be accomplished via comments on each element, each dimension, or 
overall course design. 


Structure - Context/Organization/Environm ent 


Element 

0 

1 

2 

3 

Score 

Comments 

i 

Course Design 
Framework 

No framework for 
guiding student 
throughout the 
course, navigation to 
course areas is not 
apparent 

Limited 
framework is 
apparent-the 
home page 
includes 
beginnings of 
navigation to 
course areas 

The framework for 
course delivery is 
apparent and 
includes limited 
guidance for 
navigatbn 
throughout the 
course 

Framework for 
delivery is apparent 

The home page is 
designed ta ensure 
ease of navigation 
throughout die 
course 




Course Design 
- Segmenting of 
Content, it, 
Learning Units 
or Learning 
Modules 

No modules/course 
segments exist for 
content only 
assignments are 
listed 

Several course 
modules exist, but 
progression 
between course 
modules/ 
information 
segments is not 
apparent 

Course modules 
are self-contained 
and progressbn is 
less apparent and 
not based on 
learning objectives 

Course modules are 
self-contained and 
have varying lengths 
depending on the 
learning objectives, 
with apparent 
progressbn to 
facilitate learning 



f» 

Appearance of 
Material 

Poor cobr choice - 
harsh to 

e^es/difTicult to read 

Icons are “bus/’ and 
not uniform in st^de 
and appearance 

Cobr scheme 
minimal 

Minimal 

uniformity of icon 
st^le 

Cobr scheme is 
acceptable 

Icons are 
acceptable in 
uniformity of style 

Appearance is 
appealing/ 
easy to read 

Appropriate cobr 
choice that is easy 
on the eyes 








All icons are 
uniform in style and 
cobr 
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HI 

Element 

0 

l 

2 

3 

Score 

Comments 

- Scrolling Within 

1 the Course or 
[ Within 
| Documents 

Requires extensive 
scrolling to find 
information within 
frames 

Framed 
information 
somewhat 
lengthy, requires 
more than 
minimal scrolling 

Information is 
appropriately 
framed with more 
than minimal 
scrolling required 

Information is 
appropriate ly frame d 
with minimal 
scrolling required 



I*] 

‘ J 

V 

g* 

j 

| Assignment 

1 Nav igation 

I 

Assignments within 
the course are 
difficult to find 

Assignments 
within the course 
are found with 
minimal 
searching 

Assignments are 
easily found within 
the course 

Assignments are 
easily discemable 
within the course 

A site map is 
available 



f Accessftility 

| 

There is no evidence 
that students with 
disabilities are 
accommodated in 
this course 

Minimal efforts 
are obvious 
related to 
accommodation 
of students with 
disabilities 

Limited 

accommodations 
are evident with 
some obvious 
alternative delivery 
methods 

Accommodations to 
students with 
disabilities are 
evident and 
alternative delivery 
methods are 
available 



1 Variety of 

. Assessments 
* - Assignments 
- Discussions 
| - Quizzes 
| - Surveys 

No varietyin 
assessments is 
evident 

Instructions for 
assessments are 
minimal and unclear 

At least two types 
of assessments are 
evident 

Instructions are 
provided, but are 
confusing 

At least two types 
of assessments are 
evident 

Instructions are 
provided but are 
limited 

At least three types 
of assessments are 
evident 

Clear and concise 
instructions are 
provided 







§1 

a 


Element 

0 

1 

2 

3 

Score 

Comments 

Use of Online 

Gra debook 

Online gradebookis 
not used 

Partial use of 
online gradebook; 
only some of the 
assessments are 
reflected in the 
gradebook 

All assessments are 
reflected in the 
gradebook, but 
only some grades 
are posted 

Online gradebookis 
fully used 

All ofthe 
assessments are 
reflected in the 
gradebook and all 
grades are posted 



Learning 

Resources 

No learning 
resources are posted 

Learning 
resources are 

minimal 

Learning resources 
exist indifferent 
areas but are not 
clearly articulated 
and links mayor 
may not be active 

Learning resources 
and links are 
comprehensive, 
clearly articulated, 
and current'active 



Appearance of 
Learner 

Support/ 
Feedback 
- Feedback 
Methods 

Communication 
methods with the 
students are not 
evident 

Communication 
methods exist but 
are not well used 

Communication 
methods with the 
students exist and 
are used more than 
50% of the time 

Communication 
methods with the 
students are 
comprehensive via 
accessibility to email 
and discussions 



Context for 
Learning 
Community 
where students 
interact with one 
another and the 
instructor 

- Discussions 

- Chat Rooms 

- Virtual Classes 

No evidence of the 
establishment of a 
learning community 

Learning 
community is 
established via at 
least two 
discussions 

Learning 
community is 
established™ at 
least three 
discussions 

Learning community 
is established via 
regular discussions 
and virtual 
classes/chats 
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Element 

0 

l 

2 

3 

Score 

Comments 


1 Use of 

Only one to two 

Gnlythree course 

Only four or five 

Ivbre than five 



SI 

Technology/ 

course tools are 

tools are used in 

course tools are 

course tools are used 



Course Tools, 

used in course 

course delivery 

used in course 

in course delivery 




(i.e., Email, 

delivery 

delivery 



v/ \ 

1 Asynchronous 
Discussions, 








, Synchronous Chat 
Rooms/ 







11 

Virtual Classes, 
Calendar, 

| Gradebook, 
External Links;, 
i Quizzes/Surveyjs, 







*] 

1 Group Areas, 
Student Home 








1 Pages and 
Presentations; 

^ etc.) 








| Use of 

Only one to two 

Gnlythree 

Only four or five 

Ivbre than five 



1 

Instructional 

varieties of 

varieties of 

varieties of 

varieties of 



1 Media (i.e. 

instructional media 

instructional 

instructional media 

instructional media 



a 

Ivfedia 

are used in course 

media are used in 

are used in course 

are used in course 



r 

| Presentations; 
Animated/ 

delivery 

course delivery 

delivery 

delivery 



m 

I Voiceover 







m 

| Presentations; 
Learning 

Ivbdules, Notes, 
Streaming Video, 
Internet Links, 

Case Studies, etc.) 








Structure - Context/Organization/Environment 


Comments related to structure and the development of a collaborative learning environment: 


Content - Presentation of Information 


Element 

0 

1 

2 

3 

Score 

Comments 

1 

Content of 
Learning 

Modules or 
Content of 
Learning 
Segments 

No learning 
modules/units in 

course 

Content in 
modules/units is 
inconsistent 

Lacks relevant 
material related to 
learning objectives 
and material is not 
current (resources 
used > 5 years) 

Does not have an 
introduction or a 
summary/ 
conclusion 

Limited 

consistency from 
module to module 

Resources current 
and information 
relevant to learning 
objectives 

Only an 
introduction or 
summary/ 
conclusion is 
included 

Module s/units are 
de signed and pie se nted 
in a uniform and 
consistent manner 

Resources are current 
(< 5 ^ears)and 
information is relevant 
to learning objectives 

Information presented 
is manageable with 
both an introduction 
and a conclusion 



Discussions 

- Synchronous 

- Asynchronous 

No new 
information is 
presented or ideas 
previously stated 
are not reinforced 
in either 
asynchronous or 
synchronous 
discussions/ 
interactions 

Faculty reinforces 
student ideas and 
student-pesented 
information during 
asynchronous or 
synchronous 
discussions/ 
interactions 

Faculty adds 
limited new 
information during 
asynchronous 
and/or synchronous 
discussions or 
interactions with 
students 

Faculty adds to the 
body of knowledge and 
information presented 
during asynchronous 
and/or synchronous 
discussions or 
interactions with 
students 



Links 

No liriks to Web- 
based information 
are added to the 
learning modules 
or to the course 

Minimal links are 
apparent in either 
the learning 
modules/units or 
the course;some 
are irrelevant 

An appropriate 
number of relevant 
links add to the 
learning experience 

An appropriate number 
of credible and relevant 
selected links add to 
the learning experience 
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Element 

0 

1 

2 

3 

Score 

Comments 

Course and 

Unit Learning 
Objectives 

Learning 

objective s/goals ate 
not identified 

Learning 

objectives/goals are 
identified but are 
not always 
measurable, 
behavioral, or 
appropriate in 
number for the 
content and time of 
the course 

Measurable, 
behavioral learning 
objectives/goals or 
unit objectives are 
identified for the 
course, and at least 
one learning 
objective engages 
the learner in 
activities of 
analysis and 
synthesis 

Measurable, behavioral 
learning 

objectives/goals or unit 
objectives are 
identified in the 
introduction to the 
course and the number 
is appropriate for the 
content and time for 
the course 

More than one learning 
objective engage the 
learner in activities of 
analysis, synthesis, and 
evaluation 



Course 
Assignments, 
Readings, 
Activities 
and/or Projects 

Assignments, 
activities, readings, 
and/or projects 
within the course 
are not related to 
learning objectives 

Assignments, 
activities, readings, 
and/or projects 
within the course 
canbe related to 
the learning 
objectives 

Assignments, 
activities;, readings, 
and/or projects 
within the course 
have a discussion 
of the purpose of 
the assignment 
related to learning 
objectives 

Assignment^ 
activities, readings, 
and/or projects within 
the course have a 
discussion of the 
purpose of the 
assignment related to 
learning objectives and 
are appropriate and 
manageable 



Writing Style 
(Syntax, 
Grammar, 
Punctuation & 
Flow) 

Course contains 
grammatical and 
sentence structural 

errors 

Numerous typing 
errors are present 

Basic principles of 
grammar and 
sentence structure 
are pesent 

Numerous typing 
errors are present 

Information within 
the course follows 
principles of 
grammar and 
sentence structure, 
and has few typing 
errors 

Information within the 
course follows 
principles of grammar 
and sentence structure, 
and is without typing 
errors 




Y ■ 

Sb 

0 

Element 

0 

1 

2 

3 

Score 

Comments 

Multimedia 

(Photos, 

Images, Video, 
Audio, etc.) 
and Metaphors 
Within the 
Context of the 
Content and 
Learning 
Experience 

Multimedia and 
metaphor not used 
in the presentation 
of course content 

Minimal use of 
multimedia and 
metaphor in the 
presentation of 
course content 

Multimedia used 
throughout the 
course with limited 
use of metaphor 

OR 

A progressive 
metaphor was 
developed 
throughout the 
course, with a 
limited use of 
multimedia to 
illustrate course 
content 

Multimedia used 
throughout the course 
along with a developed 
metaphor that reflects a 
piogre ssion of c ourse 
content 




Knowledge 

Limited expertise 

Inconsistent 

Expertise in 

Expertise in content 





evident in 

expertise in content 

content area 

area evident in 



■C J 


presentation of 

area evident in 

evident in 

presentation of 





content 

presentation of 

presentation of 

knowledge and in 






knowledge 

knowledge 

interactions with 







throughout the 

students 







course 





Content - Presentation of Information 


Comments related to content and the development of a collaborative learning environment: 
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Processes - Human Aspects, Relationships, Interactions, and Quality 


Element 

0 

1 

2 

3 

Score 

Comments 

1 

Interp ersonal 
Interactions 
Faculty-Student 
Student-Student 

No evidence of 
interactions student- 
to-s tilde ril and 
student-to-faculty 

Minimal 
interactions are 
evident in either 
student-to-student 
or student-to- 
faculty exchanges 

Both student-to- 
stude nt and 
stude nt-to-faculty 
interactions are 
present, but 
exchange of 
information and 
generation of ideas 
are superficial 

Dynamic and 
thoughtful 
interactions are 
evident in student- 
to-student and 
student-to-faculty 
interactions, adding 
to a quality learning 
experience 



Access to 

Faculty 
(Office Hours) 

No office hours are 
posted 

Office hours are 
posted 

Offic e hours are 
posted for both 
phone, face-to- 
face, and virtual 
times 

Timesvaiyto 
accommodate a 
variety of work 
schedules 

Office hours are 
posted for both 
phone, face-to-face, 
and virtual times 

Times vary to 
accommodate a 
variety of work 
schedules 

Schedule of office 
hours increases 
frequency prior to 
due dates of major 
course requirements 





r 

n 

F 1 

H 

[■Wjjj; 

•I] 

'J 

Element 

0 

1 

2 

3 

Score 

Comments 

Assessment 
of Learning 

Styles 

No assessment of 
learning styles is 
available 

Some information 
is provided about 
skills and 
personality 
required for 
online learning 

Specific 
information is 
provided about 
skills and 
personality 
re quired for 
successful course 
completion 

Specific information 
about skills and 
personality required 
for completion of 
the course is 
presented 

Self-assessment 
tools are available 
for the learner, and 
feedback 
information 
regarding potential 
success with online 
courses is provided 



i Instructor and 

1 Learner 

F ResponsiiOities 

1 & Guidelines for 
" Online Learning 

No guidelines are 
established for the 
learner related to 
learner and 
instructor 
responsibilities 

Only guidelines 
related to learner 
and instructor 
responsibilities 
are evident 

Some guidelines 
exist that establish 
learner and 
instructor 
responsibilities, 
online 

communication, 
and/or techniques 
to support the 
online learner;but 
guidelines are 
confusing 

Clear guidelines are 
established for the 
learner that include 
learner and 
instructor 
responsibilities;, 
online 

communication/ 
netiquette, and 
techniques to 
support the online 
learner 




Processes - Human Aspects, Relationships, Interactions, and Quality 


Comments related to processes and the development of a collaborative learning environment: 
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Outcomes - Mastery of Content and Course Evaluation 


Element 

0 

1 

2 

3 

Score 

Comments 

i 

Student Work 
Reflects 

Mastery of 

Course 

Objectives 

Student work reflects 
basic achievement of 
course objectives 

Student work 
reflects average 
understanding and 
achievement of 
course objectives 

Student work 
demonstrates above 
average 

understanding and 
achievement of 
course objectives 

Student work 
demonstrates 
masteiy of course 
content and course 
objectives 



1 

Student Work 
Reflects 

Analysis, 
Synthesis, and 
Evaluation 

Student work reflects 
basic knowledge, 
identification, or 
understanding 

Student work 
reflects 

knowledge and 
some work 
reflects analysis 
of information 

Student work 
demonstrates 
complexity with 
the majority of 
assignments below 
analysis, synthesis, 
or evaluation 

Student work 
demonstrates 
progression of 
complexity from 
knowledge to the 
level of analysis, 
synthesis, or 
eWluation for major 
assignments 



8 

Online Course 
Evaluations 

No student evaluation 
is requested for the 
course 

Student online 
evaluation input 
is requested at the 
end of the course 

Student online 
evaluation input is 
requested at the 
midpoint and end 
of the course 

Student online 
evaluation input is 
requested at the 
midpoint, end of the 
course, and after 
major assignments 
are turned in 




Learner 
satisfaction 
with the online 
learning 
experience 

No rating scale for 
learner satisfaction 

Majority of the 
learners rate the 
learning 

experience as not 
veiy satisfying 

Majority of the 
learners rate the 
learning experience 
as satisfying 

Ivhjorityof the 
learners rate the 
learning experience 
as highly satisfying 




Outcomes 


Comments related to outcomes and the development of a collaborative learning environment: 


Overall Course 


Comments related to overall course: 
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Table 1. Example of Content Evaluation 



Element 

0 

1 

2 

3 

Score 

Comments l 


jl 

1 

Y 

Content of 

Learning Modules 
or Content of 
Learning Segments 

No learning 
modules units in 

course 

Content in 
modules units is 
inconsistent 

Lacks relevant 
matenal related to 
learning objectives 
and material is not 
current (resources 
used > 5 years) 

Does not have an 
introduction or a 
summary/ 
conclusion 

Limited 

consistency from 
module to module 

Resources current 
and information 
relevant to 
learning objectives 

Only an 
introduction or 

summary 

conclusion is 
included 

Modules- units are 
designed and presented 
m a uniform and 
consistent manner 

Resources are current 
(< 5 years) and 
information is relevant 
to learning objectives 

Information presented 
is manageable with both 
an introduction and a 
conclusion 

2 

Although modules were 
used in the course, there 
was no consistency 
within the modules For 
example, two of the 
modules had learner 
outcomes and 4 did not. 

It was also noted that 3 
of the modules 
referenced matenal that 
was 8 years old. The 
references were articles 
related to research 

Discussions 

- Synchronous 

- Asynchronous 

No new 
information is 
presented or 
ideas previously 
stated are not 
reinforced in 
either 

asynchronous or 
synchronous 
discussions/ 
interactions 

Faculty reinforces 
student ideas and 
student-presented 
information during 
asynchronous or 
synchronous 
discussions' 
interactions 

Faculty adds 
limited new 
information during 
asynchronous 
andor 
synchronous 
discussions or 
interactions with 
students 

Faculty adds to the 
body of knowledge and 
information presented 
during asynchronous 
and or synchronous 
discussions or 
interactions with 
students 

3 

Excellent - the course 
faculty added relevant 
comments to the 
asynchronous 
discussion For example, 
at the beginning of each 
discussion, the faculty 
posed a number of 
questions to lead the 
discussion At the end of 
the discussion time, 
faculty summarized 
student discussions 

Links 

No links to 
Web-based 
information are 
added to the 
learning 
modules or to 
the course 

Minimal links are 
apparent in either 
the learning 
modules units or the 
course; some are 
irrelevant 

An appropriate 
number of relevant 
links add to the 
learning 
experience 

An appropnate number 
of credible and relevant 
selected links add to the 
learning experience 

3 

Numerous appropriate 
and credible links 


Element 


Score 


Comments 


Course and Unit 

Learning 

Objectives 


Learning 
objectives, goals 
are not 
identified 


Learning 
objectives goals 
are identified but 
are not always 
measurable, 
behavioral, or 
appropriate in 
number for the 
content and time 
of the course 


Measurable. 

behavioral 

learning 

objectives goals or 
unit objectives are 
identified for the 
course, and at 
least one learning 
objective engages 
the learner in 
activities of 
analysis and 
synthesis 


Measurable, 
behavioral learning 
objectives, goals or 
unit objectives are 
identified in the 
introduction to the 
course and the number 
is appropriate for the 
content and time for 
the course 

More than one 
learning objective 
engage the learner in 
activities of analysis, 
synthesis, and 
evaluation 


The course faculty 
used learning 
outcomes but a 
number of the 
outcomes were not 
measurable. For 
example, “the student 
will understand the 
policy process." 


Course 
Assignments, 
Readings, 
Activities 
and/or Projects 


Assignments, 
activities, 
readings, and or 
projects within 
the course are 
not related to 
learning 
objectives 


Assignments, 
activities, 
readings, andor 
projects within the 
course can be 
related to the 
learning 
objectives 


Assignments, 
activities, 
readings, and or 
projects within the 
course have a 
discussion of the 
purpose of the 
assignment related 
to learning 
objectives 


Assignments, 
activities, readings, 
and or projects within 
the course have a 
discussion of the 
puipose of the 
assignment related to 
learning objectives 
and are appropriate 
and manageable 


None of the 
assignments/projects 
presented a purpose 


Writing Style 
(Syntax, 
Grammar, 
Punctuation & 
Flow) 


Course contains 
grammatical and 
sentence 
structural errors 

Numerous 
typing errors are 
present _ 


Basic principles of 
grammar and 
sentence structure 
are present 

Numerous typing 
eirors are present 


Information 
within the course 
follows principles 
of grammar and 
sentence structure, 
and has few 
typing errors 


Information within the 
course follows 
principles of grammar 
and sentence 
structure, and is 
without typing eirors 


No problems 
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i* 

w, 

"1 

I" 

M 

0 

*3 

Element 

0 

1 

2 

3 

Score 

Comments 

Multimedia 

(Photos. 

Images, Video. 
Audio, etc.) 
and Metaphors 
Within the 

C ontext of the 

C ontent and 

Learning 

Experience 

Multimedia and 
metaphor not 
used in the 
presentation of 
course content 

Minimal use of 
multimedia and 
metaphor in the 
presentation of 
course content 

Multimedia used 
throughout the 
course with 
limited use of 
metaphor 

OR 

A progressive 
metaphor was 
developed 
throughout the 
course, with a 
hmited use of 
multimedia to 
illustrate course 
content 

Multimedia used 
throughout the 
course along with a 
developed metaphor 
that reflects a 
progression of 
course content 

3 

Excellent use of images, 
audio and video 
technology. The faculty 
uses a metaphor of a 
journey related to the 
course. Example, 

“research journey/ This is 
obvious throughout 
course in the use of a 
green car to remind 
students of the “journey.” 

Knowledge 

Limited 
expertise 
evident in 
presentation of 
content 

Inconsistent 
expertise in 
content area 
evident in 
presentation of 
knowledge 

Expertise in 
content area 
evident m 
presentation of 
knowledge 
throughout the 
course 

Expertise in content 
area evident in 
presentation of 
knowledge and m 
interactions with 
students 

3 

It is obvious that the 
faculty has extensive 
expertise in the subject 
area. For example, the 
faculty presents a number 
of their research reports in 
the course. Also 
interesting is a short bio of 
the faculty on the front 
page of the syllabus 


Content - Presentation of Information 


Comments related to content and the development of a collaborative learning environment: 

Although there were a few areas of concern (see scores above) the evaluator believes the faculty of record did develop a 
collaborative learning environment through the use of multimedia, links, and asynchronous discussions. 
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Table 2. Program Administrator Tracking of Online Course Evaluations 


Class Number: 

Semester 
& Year 


Semester 
& Year 



Scores 

Comments 

Scores 

Comments 

STRUCTURE | 

Course Design Framework 





Course Design - Segmenting of 

Content 





Appearance of Material 





Scrolling within the Course or within 
Documents 





Assignment Navigation 





Accessibility 





Variety of Assessments 





Use of Online Grade book 





Learning Resources 





Appearance of Learner Support/ 
Feedback 





Context for Learning Community 





Use of Technology/Course Tools 





Use of Instructional Media 





Overall Scores/Comments 






CONTENT 

Content of Learning Modules 





Discussions 





Links 





Course and Unit Learning Objectives 





Course Assignments, Readings, 
Activities, and/or Projects 





Writing Style 





Multimedia and Metaphors 





Knowledge 





Overall Scores/Comments 






PROCESSES 

Interpersonal Interactions 





Access to Faculty 





Assessment of Learning Styles 





Instructor and Learner Responsibilities 
& Guidelines 





Overall Scores/Comments 






OUTCOMES 

Student Work Reflects Mastery of 
Course Objectives 





Student Work Reflects Analysis, 
Synthesis and Evaluation 





Online Course Evaluations 





Learner Satisfaction with the Online 
Learning Experience 





Overall Scores/Comments 
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